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Abstract. New classes of non-smooth bounded domains D, for which the 
^ ^ embedding operator from into L^(D) is compact, are introduced. These 

, classes include in particular the domains whose boundary locally are graphs 

^ ■ of C— functions, but also contain much larger classes of domains. Examples 

ff~J ' of non-smooth domains for which the above embedding is compact are given. 

Applications to scattering by rough obstacles are mentioned. 

> ■ 

CN ' 1. Introduction 

, In this paper we prove some results about compactness of embedding operator 

OO ' H^{fl) LP'{Q) for rough bounded domains, that is, for domains with non-smooth 

, boundaries which do not satisfy the usual for the embedding theorems conditions, 

such as cone condition, Lipschitz domains, and extension domains (Ext-domains) . 
First, we prove compactness of the embedding operators for "elementary" domains 
' which can be approximated by Lipschitz domains in the sense described below 

II , (see the paragraph above Lemma 1.2). This class ET of "elementary" domains 

' is larger then the known classes of domains used in embedding theorems. Let us 

, give some bibliographical discussion. In jl^ a necessary and sufficient condition 

for compactness of the embedding operator is given in an abstract setting. A 
version of this result is presented in the Appendix. Compactness of the embedding 
^ ' operator for bounded domains with "segment property" is proved in |[|]. In |^ it 

was shown that the class of domains with "segment property" coincides with the 
\^ ' class C of domains whose boundaries are locally graphs of continuous functions. 

5^ \ Compactness of the embeddings for the class C was proved in |^. The reader can 

found an interesting discussion of these results in The class ET is much larger 
than the class C and includes (in the two-dimensional case) bounded domains whose 
boundaries are generated locally by the graphs of piecewise-continuous functions 
with "jump"-type discontinuity at a finite number of points. Boundaries of the 
domains of class ET can have singularities more complicated than the "jump"-type 
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singularities (see example |3.4D 



Using a Sobolev-type lemma for the union of "elementary" domains of the class 
ET we extend this result to domains of the class T which are finite unions of the 
"elementary" domains. Simple examples demonstrate that boundary of a bounded 
dom ain of class T can have countably many connected components (see example 



3.12). This is impossible for the classes of domains used in embedding theorems 



earlier (compare, for example, classes C and i5 in p| with our class T). 
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Our construction can be generalized. First, we construct a class of "elementary" 
domains with the compactness property for the embedding operator. Secondly, we 
extend this compactness property to finite unions of "elementary" domains. This 
scheme is used for quasiisometrical (the class L) and 2— quasiconformal (the class 
Q) cases. Our class L includes the Fraenkel class E. Let us explain this. Note 
that E in [9] is not extension domains. According to [9], p. 411, any domain fl of 
class E is locally -diffeomorphic at any boundary point to a domain of class C 
and dfl — dfl where fl is the closure of ft. Any domain of our class L is a finite 
union of domains that locally quasiisometrically equivalent at any boundary point 
to domains of class ET . The condition Oil — 9S1 is not necessary for the domains 
in this class. For example, if is a disc with an extracted radius, then is a 
domain of the class L, but dfl ^ dfl. 

Our class Q is much larger then the class L and includes domains with some 
"anisotropic" behavior of their boundaries (see section 4.3 for detailed explanation). 

Our results allow one to use the results in |jl^ and |Q on the existence and 
uniqueness of the solutions to the scattering problem in the exterior of rough ob- 
stacles and consider larger class of rough obstacles in scattering theory than it was 
done earlier. 

2. Abstract result 

In this section we prove some results which give conditions for the compactness 
of an embedding operator, and use these results in a study of compactness of 
the embeddings of Sobolev spaces. An abstract necessary and sufficient condition 
for the compactness of an embedding operator is proved in Let Hi and H2 

be Hilbert spaces and Hi C H2 . Here the embeddings are understood as the 
set-theoretical inclusions and the inequalities ||u||i > ||m||2 are assumed, where 
ll^llj ■= Ikllffj- Suppose that Ts , s e (0, 1) is a family of closed subspaces of H2 
and To- C Tg for s < a. 

In our applications Ts = L'^{Ds), where Dg C D, (Z for s < a. We assume 
below (see lemma 1.2) that a domain D, for which we study the compactness of the 
embedding operator from H^{D) into L^{D), contains a Lipschitz subdomain Gg, 
Ds C Gs C D. Let Ps be the orthogonal projection onto Ts in H2, i ■ Hi ^ H2 
be the embedding operator, and is ■= Psi- Let us state two results. The above 
assumptions and notations are not repeated. The first result is obvious. 

Proposition 2.1. // the operator i : Hi ^ H2 is compact, then the operator is is 
compact for any s G (0, 1) . 

Proof. If i: Hi H2 is compact, then is is a composition of a bounded linear 
operator Ps and a compact operator i, so is is compact. □ 

The following proposition is used in the proof of proposition 1.4 below. 

Proposition 2.2. If the following conditions hold: 

1) is is compact for all s G (0, l)and 

2) ||u||2 < a(s)||M||i -I- 6||Psw||2, a(s) > 0, linis^o a(s) = 0, 6 > for any u e 
Hi, then the embedding i : Hi ^ H2 is compact. 

Proof. Choose a sequence {sm} such that a(s„i) < ^. Denote by Pm the projection 
Ps^. If is is compact then || Un ||i= 1 impHes || Ps{un) ||2< 1 and for any m there 
exists a subsequence Un,m and a number n(m) such that || Pm(un,m — Uni,m) Ii2< ^ 



COMPACTNESS OF THE EMBEDDING OPERATORS FOR ROUGH DOMAINS. 



3 



for any n, ni > n{m). Without loss of generality we can suppose that the sequence 
Un,mi is a subsequence of u„,m and n(rn) < n(mi) for m < mi . Therefore 

II Pm{'^n(m),m ^n(mi),m) ||2^ 

m 

for any m and for any mi > m . 

For the subsequence Um ■= w„(m),„i condition 2) implies || Um — Umi |l2< a(sm) || 
Um — "mi 111 +b II Pm{um — ""mi) ||2- By choice of the subsequence {um\ this 
implies || Um — ||2< (1 + for any m . We proved convergence of {um} in 
H2. Because the original sequence || u„ ||i= 1 was arbitrary compactness of the 
operator i proved. 

□ 

We apply this abstract result to Sobolev spaces. Suppose that D C i?" is 
a bounded domain and {Dg}, < s < 1, is a family of subdomains such that 
Ds C Da for any s < a and for any s there exists a Lipschitz domain Gs such that 
Ds C Gs C D. A bounded domain is a Lipschitz domain if its boundary is locally 
graph of a Lipschitz function. 

Lemma 2.3. Suppose that {un} is a bounded sequence in H^{D). Then there 
exists a subsequence {wnt} of the sequence {«„} which converges in L^{Ds), i.e. 
is : H^{Ds) L'^{Ds) is compact for all s G (0, 1). 

Proof. One takes the Lipschitz domain Gg such that Dg C Gg C D. By the known 
embedding theorem for Lipschitz domains the embedding H^{Gs) — + L?'{Gg) is 
compact. Since Dg C Gg C D, one obtains the conclusion of the lemma. □ 

Proposition 2.4. If the following condition hold 

\\u\\l^(d) < a{s)\\u\\Hi{D) +b\\u\\L^tD,), a(s) > 0, lim a(s) = 0, b > 0, 

for any u e H^{D). 

Then the operator i : H^{D) L'^{D) is compact 

By Lemma 2.3 ig : H^{Dg) L'^{Dg) is compact for all s E (0, 1). Hence the 
claim follows from Proposition 2.2. 

In section 3-4 we describe classes of domains for which the conditions of the 
proposition 2.4 are satisfied. 

3. Domains of class T. 

Below we denote a domain by fl. The main purpose of this section is to prove 
compactness of the embedding operators H^{fl) — L^{fl) for domains of the class 
T which we describe below. Domains of the class T are finite unions of elementary 
domains of the class ET whose boundaries are locally graphs of "good" functions: 
these domains can be approximated by Lipschitz subdomains in such a way, that 



conditions of Proposition 2.4 hold. For example, in the two-dimensional case the 
function is "good" if it is piecewise-continuous with discontinuity points of "finite 
jump" type. 

In the first part of this section we describe exactly classes T and ET. In the 
second part we derive an auxiliary one-dimensional inequality. This inequality is 
not new, but its proof is. It is a version of Agmon inequality M adopted for our 
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purposes. In the final part of tfiis section we prove compactness of tiie embedding 
operator for domains of class T using the results of section 2. 

3.1. Preliminaries. Let x G R", x = (a;i, a;2, and Qn = [0,1]" be the 

standard closed cube in i?". Denote x := {xi,X2, ...,.t„_i). 

A bounded function / : Qn-i ^ i? is an admissible function if / is continuous 
on a set C{f} C Qn-i such that n{Qn-\ \ C{f)) = 0, where is n — 1-dimensional 
Lebesgue measure. Denote by IntA the set of all interior points of A.. 

Definition 3.1. Wc call U := Int{Qn + (0, 0, /(a;'))} a standard elementary 
domain if the function / is admissible. 

Let Qn,h = [h,l — /i]" be a subcube of the standard cube Qn- If U := Int{Qn + 
(0, 0, f{x'))} is an elementary domain then denote 
Uh = Int{Qn,h + {0,-,0,fix'))}. 

Definition 3.2. We call a standard elementary domain U a standard elementary- 
domain of class ET if for any < h < 1/3 there exists a Lipschitz domain Vh such 
that Uh C Vh C U. We call U an elementary domain of class ET if it is an image 
of a standard elementary domain of class ET under a afEne invertible mapping of 
i?" onto E". 

Example 3.3. Suppose that / : [0, 1] ^ i? is a piecewise continuous bounded 
function with finite number of discontinuity points 2:1,2:2, .-^Xk and at any discon- 
tinuity point the function / has right and left limits (i.e. any discontinuity points 
are "jump" points). The domain U := Int{Q2 + (0, f{x))} is a standard elementary 
domain of the class ET. 

Proof. It is obvious that U is a standard elementary domain. Fix < ft- < 1/3. 
The open set Wh = Int{U \ Uh) is a finite union of domains Ui — {xi^i,Xi) x (1 — 
h + f{x), 1 + /(2;)) and domains Vi = {xi-i,Xi) x (/(x), h + f{x)), i = I, fc + 1, 
Xf) = a, Xk+i = b. Join any two points (xi_i, 1 — h/2 + lim^^^+ ^ f{x), (x^, 1 — 
ft./2 + lim^^^- f{x) by a smooth curve Ui and any pair (xi_i, ft/2 + lim^^^+ ,f{x), 
{xi, h/2 + hm^^^- f{x) by a smooth curve A . The set dU \ dWu U (u|ri a^) U 

(uf^i^A) is a closed Lipschitz curve that is the boundary of a Lipschitz domain Vh- 
By construction Uh C Vh C U - Therefore f/ is a standard elementary domain of 
the class ET. □ 

Example 3.4. Suppose that / : [0, 1] —> J? is a piecewise-continuous bounded 
function with countably many isolated discontinuity points Xi,X2, .-jXk, ... and at 
any discontinuity point the function / has right and left limits (i.e. any discontinuity 
points are "jump" points). Suppose also that the sequence {xk} converges to xo- 
The domain U := Int{Q2 + (0, f{x))} is a standard elementary domain of class 
ET. 

Proof. Because / is continuous in xq for any < /i < 1/3 the open set Wh = 
IntU \ Uh is a. finite union of domains of the same type as in example 3.3. Therefore 
the domain U := Int{Q2 + (0, f{x))} is a standard elementary domain of the class 
ET. 

□ 
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3.2. One-dimensional inequality. 
Lemma 3.5. If u ^ H^{{—h,h)), then 

\\\u\\l^{{OM)) ~ \\u\\L^({-hM))\ < V2h.\\-j^\\L^{{~h,h))- 

Proof. Since smooth functions are dense in H^{{—h,h)) it is sufficient to prove 
the desired estimate only for smooth functions u e H^{{—h,h)). Integrating the 
inequahty \u{t + h) - u{t)\^ < {J^+'' \^{s)\ds)^ < {j'W^{s)\ds)^ with respect to 
t over the segment [~h, 0] and using the Holder inequality we obtain 



\u{t + h)-u{t)\^dt<h{ \ — {t)\dtY<2h^ \ — {t)Ydt. 
h J-h "-t J_i^ dt 

For any normed space X and any x,y £ X the following inequality holds 

Ilkll-|l2/lll<lk-y||- 

Combining this inequality with previous one we obtain 

j-O pO m 

\{ \u{t + h)\^dty/^ -{ \u{t)\^dty/^\ < { \u{t + h)-u{t)\^dt)^^^ < 



Because \u{t + h)\dt — J^^ \u{t)\dt we have finally 



|w||l2((o,/i)) - ||m||l2((-/i,0))I < '^H-^\\LH{-h,h))- 



Corollary 3.6. If u e H^{{~h,h)), then 

\u{t)\^dt<2 f \u{t)\^dt + Ah^ r \^{t)\^dt, 

J-h J-h dt 



\u{t)\^dt<2 \u{t)\''dt + Ah^ \^{t)\''dt. 

Jo J-h dt 



□ 



Proof. Using Lemma 3.5, one gets: 



rill 

Ht)\'dt < [ / \u{t)\'dt)^/' + V2h{ J J^(t)|2dt)l/2]2 < 



J-h 



2 1 \u{t)\^dt + 4h^ r \^{t)\^dt. 



-h 



-h dt 



□ 
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Proposition 3.7. If u ^ H^{{a,b)), then 



Ja+h Ja "-t 



for any h < 

Proof. By the previous corollary 



\u[t)\^dt< / \u{t)\'^dt+ / \u{t)\^+ / \u{t)\^dt< 

Jb-h Ja+h Ja 

pb—h pb—h i*a+2h 

2/ \u{t)\^dt+ \u{t)\^ + 2 \u{t)\''dt+ 

Jb-2h Ja+h Ja+h 

dv dv 

^h' At)\^dt + Ah' \^{t)\'dt< 

Ja dt Ji,_f, dt 

3 /' \u{t)\^dt + ^h^ f \'^{t)\^dt. 



a-\-h 



dt 



□ 



3.3. Compactness for elementary domains of class ET . 



Proposition 3.8. If U = {a,b) x F/ is an elementary domain of the class ET , 
then the embedding operator i : H^{U) L'^{U) is compact. 

Proof. It is sufficient to prove this proposition for a standard elementary domain 
of class ET. 

Fix /i < i and choose a sequence {u„} C H^{U), ||wn|j/fi(;7) < 1 for all n. 
Since H^{U) is a Hilbert space, one may assume without loss of generality that the 
sequence {«„} weakly converges in II^{U) to some function uq G H^{U). 



Using Proposition 3.7 for almost all x' in the domain of definition Qn-i of 



continuous function / we get 

l + f{x') i-f{x') + l~h 

\Un{x',t) - UQ{x',t)\'^dt <3 / \un{x',t) - uo{x',t)\'^dt 



fix') Jf{x')+h 



,2 f' + f'^^'\d{Un-Uo). , ,,2 



+4/1^ / r^-- -' {x\t)\'dt. 

J f(x') "t 

Integrating this inequality over Qn-i we obtain 



\un{x) — uq{x)\ dx < 3 \un{x) — uo{x)\ dx 

U Juh 



+4/iM |V(m„ - uo)(a;)| dec. 
Ju 

Therefore all conditions of Proposition hold and the embedding operator i is 
compact. 

□ 



COMPACTNESS OF THE EMBEDDING OPERATORS FOR ROUGH DOMAINS. 



7 



3.4. Compactness for domains of class T. 

Definition 3.9. A domain fl belongs to class T if it is a finite union of elementary 
domains of class ET . 

First, we prove a Sobolev type lemma for compact embeddings. 

Lemma 3.10. Letfli andQ,2 he such domains that embedding operators H^{Qi) 
L'^{fli) and H^{fl2) L'^{fl2) are compact, then the embedding operator H^{fli U 
fl2) ^ L^{fli U fl2) is also compact. 

Proof. Choose a sequence {wn} C H^{fli U ^^2), ||wn||Hi(siiusi,) < 1 for all n. Let 
M„ := m„|rii and := w„|ri2- Then u„ G -ff^^i) , e H^{fl2), ||w„||hi(Oi) < 1, 
\\vn\\m{n^) < 1- 

Because the embedding operator iJ^(rii) L^{Qi) is compact we can choose a 
subsequence {un^} of the sequence which converges in L^(Oi) to a function 

uq S L^(r2i). Because the second embedding operator H^{fl2) ^ L^i^2) is also 
compact we can choose a subsequence {vuk^ } of the sequence {vn^} which converges 
in L^(ri2) to a function vq € L^(02). It is evident that uq — vq almost everywhere 
in fli n fl2 and the function wo{x) which is defined as wo{x) := uq{x) on fli and 
wo{x) :— vo(x) on ^2 belongs to L^(fli U fl2). 

Hence 



Therefore - wo||L2(niun2) ^ for m 00 . □ 



From Proposition 3.8 and Lemma B.IO the main result of this section follows im- 
mediately: 

Theorem 3.11. If a domain fl belongs to class T then the embedding operator 
H^{Q) — > L^(ri) is compact. 

The example below demonstrates the difference between class T and the class 
of bounded domains whose boundaries are locally graphs of continuous functions 
(C-domains). The boundary of a domain of class T can have countably many 
connected components, while this is not possible for C-domains. 

Example 3.12. Take: U :— {{xi,X2) : < xi < I/tt, xi sin — < X2 < xi sin — -|- 
4}; y = (0, I/tt) X (-2,0) , 17 = [/U 

Domains U and V are elementary domains of class ET. Therefore is a domain 
of class T . By Theorem 3.11 the embedding operator H^{fl) L^(ri) is compact. 

Let us discuss the structure of dfl. The boundary dU is connected and contains 
the graph Tf — {{xi,X2) ■ X2 = a;i sin of the function / : [0, i] R, f{xi) — 
xisin^^. The graph F/ can be divided on two parts: the "nonnegative" part 
Fj :— {{xi,X2) C r f : X2 > 0} and "negative part TJ :— {{xi,X2) C T j : X2 < 0}. 
The "negative" part TJ C V. Therefore the boundary of the plane domain Q 
does not contain TJ and consists of the countably many connected components: 
Si = ([0,l/7r]x{-2})U({0}x(-2,4))U({i}x(-2,4))UFg, where F^ is the graph of 
the function 5 : [0, i] ^ i? , g{xi) = Xi sin ^ + 4; = {[t^^, W^^] ^ {0})UF, 
i = 2, Ti C TJ is the graph of the restriction of the function f{xi) = xi sin ^ to 
the segment [(2131)7' 2(1-1)77 ]' ^ ~ ^ point of the boundary dil. 
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Notice that any neighboorhood of the point {0,0} the boundary dfl has count- 
ably many connected components and therefore can not be presented as a graph of 
any continuous function which is a connected set. 

Higher-dimensional examples can be constructed using the rotation of 
two-dimensional domain around xi-axis. 

The following corollary is practically convenient for using the main theorem. 

Corollary 3.13. // a bounded domain U is an extension domain, a domain V 
belongs to class T and Vl := U yjV , then the embedding operator H^{Vl) i^(ri) 
is compact. 



This corallary follows from Theorem 3.11 and Lemma 3.10 



Example 3.14. Let U := U{f,g,XQ,r) {{x ,Xn) ■ g{x ) < x„ < f{x )} where a 
continuous real-valued functions f,g defined on the closed ball B :— Bn-i{xQ, r) C 
and H := max^>^-g{f{x ) — g{x )) > 0.. Then the embedding operator 
H^{U) L^{U) is compact. 



The above claim follows from corollary p.l3| . We need only to represent C/ as a 
union of domains of class C and an extension domain (in our case a domain with 
Lipschitz boundary). 

Remark. Extension domains can have very rough boundary. In the plane a bounded 
domain U has an extension property if and only if it is an image of the unit disc 
under quasiconformal homeomorphism (}> : (see For example 

the Hausdorff dimension of an image dU of a unit circle under quasiconformal 
homeomorphism (jj : R^ ^ R"^ can be any number 1 < a < 2 |Q. 

4. QUASIISOMETRICAL HOMEOMORPHISMS AND COMPACT EMBEDDINGS. 

A large class of bounded domains in i?" does not belong to class T but still have 
"good" properties like compactness of the embedding H^{Q) L^{Q). To study 
these domains we will introduce a larger and more fiexible class of "elementary" do- 
mains, i.e. quasiisometrical images of elementary domains of class ET. Then we ex- 
tend the main theorem to the finite unions of quasiisometrical elementary domains. 
Our proof is based on the well-known fact that a quasiisometrical homeomorphism 
(f : U ^ V induces a bounded composition operator ip* : H^{V) — > H^{U) by the 
rule (p*{u) — uo ip (see, for example ||^ or ||l^ ). 

Recall the definition of a quasiisometrical homeomorphism. 

Definition 4.1. Let U and V be two domains in R". A homeomorphism ip : U ^ 
V is Q— quasiisometrical (or simply quasiisometrical) if for any point x G U there 
exists such a ball B{x,r) C U that 

(4.1) Q-'\y - z| < \^{y) - ^(z)| < Q\y ~ z\ 

for any y,z £ B{x, r). Here the constant Q > does not depend on the choice of 
x&U. 

Obviously the inverse homeomorphism (^~^ :V ^ U \& also Q— quasiisometrical . 
Domains U and V are quasiisometrically equivalent if there exists a quasiisometrical 
homeomorphism if : U V . 

Any quasiisometrical homeomorphism is a locally bi-Lipschitz, weakly differen- 
tiable and differentiable almost everywhere. 
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Any diffeomorphism tp : U ^ V is quasiisometrical on a subdomain Ui d U if 
the closure Ui of Ui belongs to U . 

Let us demonstrate a practical way to construct a new quasiisometrical home- 
omorphism using a given one. Suppose that Sk{x) — kx is a similarity transfor- 
mation (which is called below a similarity) of i?" with the similarity coefficient 
k > 0, Ski{x) — kix is another similarity and ip : U V is a Q— quasiisometrical 
homeomorphism. Then a composition ip := Sk ° p> o Ski ^ kikQ -quasiisometrical 
homeomorphism . 

It is easy to check this claim. Because ip : U ^ V is Q— quasiisometrical for any 



point X gU there exists such a ball B{x,r) C U that the inequality 2.2 holds. 
Therefore 

IV'(y) - '^{^-)\ = Mfikiy) - (p{kiz)\ < kQ\kiy - kiz\ < kikQ\y - z\ 
for any y,z £ k^^ B{ki^x, ki^r). By the same reasons 

|7/;(y)-^(z)| > (fcifcQ)-i|y-z|. 

If ki = k^^ then the homeomorphism i/j is Q— quasiisometrical. 

This remark will be used in example 2.2 of a domain with "spiral" boundary 
which is quasiisometrically equivalent to a cube. We start with a two-dimensional 
Wexample. 

Example 4.2. We will construct a domain with "spiral" boundary with the help 
of a quasiisometrical homeomorphism. We can start with the triangle T := {(s, t) : 
0<s<l,s<t< 2s} because T is quasiisometrically equivalent to the unit 
square Q2 — (0,1) x (0,1). Hence we need to construct only a quasiisometrical 
homeomorphism ipo from T into R^. 

Let {p,0) be polar coordinates in the plane. Define first a mapping (p : 
as follows: <p{s,t) = {p{s,t),9{s,t)), p{s,t) = s , 0{s,t) = 27rln^. Here 
i?^ :— {(s,i)|0 < s < 00,0 < t < 00}. An inverse mapping can be calculated 
easily: ip~^{p,9) — {s{p,6),t{p,6)), s{p,9) — p, t{p,9)) = p'^e^ . Therefore (p and 
Lpo — Lp\T are diffeomorphisms. 

The image of the ray i = A:s,s>0,fc>Ois the logarithmic spiral p — 
kexp{—j^). Hence the image S :— (p{T) = ^o{T) is an "elementary spiral" 
plane domain, because dT is a union of two logarithmic spirals p = exp(— ^), 
p = 2exp(— ^) and the segment of the circle p = 1 . 

The domain T is a union of countably many subdomains r„ := {(s, t) : e~*^"+^-' < 
s < e~("~^\ s < t < 2s}, n = 1, 2, ... . On the first domain Ti the diffeomorphism 
ipi := ip\Ti is Q— quasiisometrical, because ipi is the restriction on Ti of a diffeo- 
morphism ip defined in i?^ and Ti C We do not calculate the number Q. 

If we will prove that any diffeomorphism := ip\Tn is the composition ipn — 
S'g_(„_i) o ipi o Sgn-i of similarities S'g-(„-i), S'e^-i and the Q— quasiisometrical 
diffeomorphism (pi, then any diffeomorphism ipn is Q— quasiisometrical, the dif- 
feomorphism (fio is also Q— quasiisometrical, and the "elementary spiral" domain 
U = <po{T) is quasiisometrically equivalent to the unit square. 

Let us prove the representation (fn = Sf,-(n-i) o cpi o Se^-i. 

By construction the domain Ti is the image of T„ under the similarity transfor- 
mation ^eii-i (s, t) = e"~^(s, t). Therefore we need to prove only the representation 
(fi = S'g-(n-i) o ip o Sf.n~i. This representation follows from a direct calculation: 

(5,-(„-i, o <p o 5,„-i )(s, t) = 5,-(„-i, (p(e"^is, e"~4), 0{e""h, e"~4)) 



COMPACTNESS OF THE EMBEDDING OPERATORS FOR ROUGH DOMAINS. 



10 



= (e-("-^V(e""^s,e"~4),6'(e"-^s,e"-4)) = (s, 27r In — - 27r(n - 1)) 



Remark. By a rotation we can construct corresponding higher-dimensional exam- 
ples of domains with "spiral" type singularities. 

4.1. Domains of class L. 

Definition 4.3. A domain U is an elementary domain of class L if it is a quasi- 
isometrical image of an elementary domain of class ET . 

A domain U is a domain of class L if it is a finite union of elementary domains 
of class L. 



Proposition 4.4. (see for example ^ or Let U and V be domains in i?". 
A quasiisometrical homeomorphism (p : U ^ V induces a bounded composition 
operator tp* : H^{V) ^ H^{U) by the rule ^*(u) =uo ip. 



Combining this result with Theorem |3.11| and Lemma |3.10| we obtain 



Tlieorem 4.5. If a domain fl belongs to class L then the embedding operator 
H^{Q) i^(ri) is compact. 

Proof. Let U be an elementary domain of class L. Then there exists an elementary 
domain V of class ET and a quasiisometrical homeomorphism ip : V ~* U. By the 
previous theorem operators ip* : H^iV) ^ H^iU) and (i,^"^)* : H^{U) E[^iV) 
are bounded. By Theorem [s!^ the embedding operator ly ■ H^{V) L^iV) 
is compact. The embedding operator lu : H^{U) L'^{U) is the composition 
{ip~^)* o ly o ip* . Therefore the embedding operator lu : H^{U) L'^{U) is 
compact. 

Because any domain fl of class L is a finite union of elementary domains of class 
L the result follows from Lemma 3.1C| . □ 



5. Domains with nonlocal singularities of the boundaries 

The previous section focuses on domains which are locally quasiisometrical im- 
ages of domains of class T. For the proof of the main result we used the compactness 
of embedding operators for domains of class T and the boundedness of composition 
operators induced by quasiisometrical homeomorphisms. 

In this section we use similar arguments for the largest class of homeomorphisms 
that induce bounded composition operators of the Sobolev spaces H^. 

We recall the main idea for a study of the embedding operators proposed in |^. 
Let SI be a domain with "good" boundary, for example, domain of class L, and U 
be a domain with "bad" boundary. Suppose that there exists a homeomorphism 
(/) : Q —f U such that (j) induces a bounded composition operator 0* : H^{U) — > 
H^(p.) by the rule 4>*{u) = uo ip and 4>~^ induces a bounded composition operator 
: L'^in) L'^iU). If the embedding operator In : H\n) L^{n) is 
compact, then the embedding operator Ijj = Iqc/)* '■ H^i^) ^ L'^{fl) is also 

compact. 

This method was used in |||] for a study of the embedding operators in domains 
with "nonlocal" singularities. 
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5.1. 2-quasi-conformal homeomorphisms. Composition operators for Sobolev 
spaces with first generalized derivatives were studied in detail in 0. We restrict 
ourselves to the practically important class of locally bi-Lipschitz homeomorphisms. 

Definition 5.1. A locally bi-Lipschitz homeomorphism (p : Q ^ U is 2-quasi- 
conformal if there exists a constant K such that 

\\<t>'{x)f < K\det(l)'{x)\ 

for almost all x £ fl . The 2-quasi-conformal dilatation K{(j3) is a minimal number 
K for which the previous inequality holds. 

Here 4>'{x) = \^^{x)\,i, j = l,2,..,n is the Jacobi matrix of the mapping Lp at 

the point x and ||(/)'(a;)|| :— ^X]"j=i(if^(^))'^ is the norm of the Jacobi matrix. 

Obviously any quasiisometrical homeomorphism is 2-quasi-conformal. Composi- 
tion of 2-quasi-conformal homeomorphisms is 2-quasi-conformal . 

Choose two bounded domains 17, [/ in i?" , n > 2 . 

Theorem 5.2. (see locally bi-Lipschitz homeomorphism (j) : ^ U induces 

a bounded composition operator (j)* : H^{U) — > H^(fl) if and only if (j) is 2-quasi- 
conformal. 

This result was used in the following version of the so-called "relative" embedding 
theorem. 

Theorem 5.3. (see ^) Suppose that a homeomorphism (p : ft ^ U is 2-quasi- 
conformal and II det (/)'(x)||i<xj(Q-) < oo . If the embedding operator In : 11^(0.) 
L^(0) is compact then the embedding operator lu : II^(U) — > L'^{U) is also compact. 

The following corollary helps to use this result practically: 

Corollary 5.4. Suppose that Q. is domain of class L and there exists a 2-quasi- 
conformal homeomorphism (j) : Q, ^ U . If \\ dei (j)' {x)\\ i^a^ (^^-i^ < oo, then the embed- 
operator Ijj : II^{U) L'^{U) is compact. 



This corollary follows immediately from the previous theorem and the embedding 
theorem for T— domains. 

It allows one to use the method of Section 2 for 2-quasi-conformal case. 

5.2. Domains of class Q. 

Definition 5.5. A domain U is an elementary domain of class Q if there exist an 
elementary domain V of class L and a 2-quasi-conformal homeomorphism ip : U ^ 
V such that || det (/)'(x)||ioo(Q-) < oo. 

A domain U is a domain of class Q if it is a finite union of elementary domains 
of class Q. 



Combining Corollary 5.4 with the Theorem 4.5 and Lemma 3.1C we obtain 

Theorem 5.6. If a domain fl belongs to class Q then the embedding operator 
L^{fl) is compact. 

Proof. Let U be an elementary domain of class Q. Then there exists an elementary 
domain V of class L and a 2-quasiisomet rica l homeomorphism (p : V ^ U such 
that II det0'(a:)||Loo(o) < cx3. By Theorem U operators ip* : H^{V) H^{U) and 
{(fi^^)* : H^{U) H^iy) are bounded. By Corollary 5.4 the embedding operator 
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Iv ■■ H^{V) L'^iV) is compact. The embedding operator lu : H^{U) -> L'^{U) 
is equal to the composition i^^^)* o ly o ip* . Therefore the embedding operator 
lu ■■ H\U) L^iU) is compact. 

Because any domain of class Q is a finite union of elementary domains of class 



the result follows from Lemma 3.1C. □ 



Let us demonstrate a simple example of an elementary domain of class Q with "non 
local" singularity near the point {0}. 

Example 5.7. Let Q € R"^ he the union of rectangles Tk = {x € R'^ : jxi — 2~"'^| < 
2-a(fe+2).o < < 2-"('=+2)},0 < a and the square Q = (0,1) x (-1,0) . It is 
easy to check that the homeomorphism (f{xi,X2) = (a;i|x| a;2|a;| ° is 2-quasi- 
conformal and fii = (p{^) is the union of rectangles Pk = {x £ R'^ : \xi — 2^^ < 
2-(/c+2).Q < X2 < 2-('^+2)} < a and the square Q = (0,1) x (-1,0) . In [|o| 
a quasiisometrical homeomorphism from ili to the unit square is constructed. 
Hence the composition </) = "0 ° is a 2-quasi-conformal homeomorphism and by 
direct calculation we can check that || det 0'(a;)||ioo(f^) < oo. Therefore the domain 
fl is an elementary domain of class Q. 

Remark that a projection of B{0, r) C\dVl into arbitrary line L S i?^ is not one to 
one correspondence for any r and L . Therefore the domain Q. is not an elementary 
domain of class C. 

Higher-dimensional examples can be constructed using rotations. 

5.3. Discussion of 2-quasiconformal homeomorphisms and 
2-quasi-conformal domains. Let us give first a geometrical interpretation of 
2-quasi-conformality. 

Suppose that (p : R" ^ i?" is a Hnear homeomorphism, ip' is its matrix and (0')^ 
its adjoint matrix. Denote by < X2 — ■■■ ^ eigenvalues of [(p'Y'cf)' . There 
exist two orthogonal bases 61,62, ...,6„ and 51,32, such that 4>{ei) — XiQi for 

every i = 1,2, ...,rt. Geometrically Xi is length of i—th semi-axis of the ellipsoid 
0(i?(0, 1)) . The 2-quasi-conformal dilatation K{(/)) — ^^^^'^"^ — -. 

If (/? : — > L/ is a diffeomorphism then the numbers Ai(a;) < A2(.2;) < ... < Xn{x) 
correspond to the linear homeomorphism dcf) and 

i^W-sup[ ]. 
xen Xi(x)X2{x)...Xn-i{x) 

If : — > J7 is only locally Lipschitz then 

Xnix) 



Ai(x)A2(x)...A„_i(x) 



The relations of 2-quasi-conformal homeomorphisms with the traditional classes 
can be described as follows: 

1) In the two-dimensional case 2-quasi-conformal homeomorphisms are quasi- 
conformal. A homeomorphism inverse to a quasi-conformal homeomorphism is 
also quasi-conformal. Therefore a homeomorphism inverse to 2-quasi-conformal 
homeomorphism is 2-quasi-conformal (for plane domains). Unfortunately, this 
property does not hold in the higher-dimensional cases. In |0| an example of 2- 
quasi-conformal homeomorphism with non-2-quasi-conformal inverse homeomor- 
phism is constructed. Composition of 2-quasi-conformal homeomorphisms is a 2- 
quasi-conformal homeomorphism. 
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2) Two-dimensional conformal mappings are 2-quasi-conformal homeomorphisms 
with = 1. 

3) Any quasiisometrical homeomorphism is 2-quasi-conformal. 

Appendix 

In this section an abstract necessary and sufficient condition for the embedding 
operator to be compact is given. In our presentation the work iQ is used. 

Let Hj, j = 1,2,3, be Hilbert spaces, Hi C H2 C H^, the embeddings mean 
set-theoretical inclusions and the inequalities ||w||i > ||u||2 > l|w||3, where \\u\\j := 
llull//^. This implies the compatibility of the norms: 

if llunlla and ||u„ — m||2 ^ then u — 

Denote by i the embedding operator from Hi into H2 and by j the embedding 
operator from Hi into H3. 

Proposition 5.8. The operator i : Hi ^ H2 is compact if and only if the following 
conditions hold: 

1) i is compact, 
and 

2) \\u\\2 < -I- c(e)||u||3 for all e G (0,eo), c(e) = const > 0, for all u £ Hi 

Proof. Necessity: condition 1) is clearly necessary: if i: Hi H2 is compact, and 
H2 C H3, \\u\\2 > ||u||3, then j: Hi H^ is compact. 

To prove 2), assume the contrary: there exists u„ G Hi, ||u„||i = 1, and e G 
(0, eo) such that 

||u||2 > + n||u||3 

for all n = 1, 2, .... 

Since ||wn||i — 1 > ||un||2, one concludes from previous inequality that ||wn||3 — * 
as n — > 00 and u„ ^ u in ^ stands for weak convergence. Since i: Hi H2 
is compact, it follows that ||w„ — m||2 0. Since ||un||3 ^ it follows that u = and 
llwnib 0. This is a contradiction: by condition 2) the inequality ||u„||2 > e > 
holds. Necessity of 1) and 2) is established. 

Sufficiencv: if j is compact then Hitnlli = 1 implies that a subsequence u„ (denoted 
again m„) converges in H3, that is — Um||3 ^ as n,m ^ 00. Condition 2) 

implies ||u„ - Mm||2 < s\\Un - Um\\l + c(£)||m„ - Um||3- 

Fix an arbitrary small S > 0. Note that ||un — Mm||i < 2. Choose e = (5/4 and fix 
it. Then choose n,m so large that c(£)||w„ — Wmlls < <5/2. Then ||w„ — itmlb < <5. 
This implies convergence of m„ in H2. The sufficiency is proved. □ 

Acknowledgement: AGR thanks Prof. V.Maz'ya for useful correspondence. 
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